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The Problem 


• Satellite conjunction risk typically evaluated through the probability 
of collision (Pc) 

-Considers both conjunction geometry and uncertainties in both state estimates 

• Conjunction events initially discovered through JSpOC screenings, 
usually seven days before Time of Closest Approach (TCA) 

- However, JSpOC continues to track objects and issue conjunction updates 

-Changes in state estimate and reduced propagation time cause Pc to change 
as event develops 

-These changes a combination of potentially predictable development and 
unpredictable changes in state estimate / covariance 

• Operationally useful datum: the peak Pc 

- If it can reasonably be inferred that the peak Pc value has passed, then risk 
assessment can be conducted against this peak value 

- If this value below remediation level, then event intensity can be relaxed 

• Can the peak Pc location be reasonably predicted? 


• I a.i. solutions 
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Conjunction Event “Canonical Progression” 


• Conjunction typically first discovered 7 days before TCA 

-Covariances large, so typically Pc below maximum 

• As event tracked and updated, changes to state estimate are 
usually relatively small, but covariance shrinks 

- Because closer to TCA, less uncertainty in projecting positions to TCA 

• Theoretical maximum Pc encountered when 1 -sigma covariance 
size to miss distance ratio is 1/V2 

-After this, Pc usually decreases rapidly 

• Behavior shown in graph at right 

-X-axis is covariance / miss distance 
-Y-axis is log 10 (P c /max(P c )) 

- Order of magnitude change in Pc considered 
significant, thus log-space more appropriate 

• How might this behavior be modeled? 

- Underlying progression in presence of noise 

a.i. solutions 



Ratio of 1 -sigma Covariance Radius to Miss Distance 
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Proposed Choice of Modeling Variables 



• Dependent variable is loglO value of Pc 

- Need to address problem of very small and 0 values for Pc 

- Majority of Pc values for purposes of operations “essentially 0”: < IE-10 

• Small values of Pc can be “floored” at IE-10 

• Furthermore, long trains of leading or trailing IE-10 values can also be eliminated 
from dataset for model tuning and evaluation; really just a function of when updates 
happen to occur. 

• Independent variable is time before TCA (usually in fractional days) 

-Canonical behavior curve uses independent variable as ratio of covariance 
size to miss distance 

- Problematic independent variable for fitting 

• Not monotonic with time (but it does correlate at least moderately to time) 

• Need temporal independent variable in order to map to operational timelines 

-Thus, use time before TCA as independent variable for model 


• ft a.i. solutions 
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Bayesian Vertex Model 


• Approximate theoretical progression of log(P c ) values using a 
downward-opening parabola 

- Equation in vertex form: Y = a (x - h ) 2 + b 

- Can be recast as: Y = p 0 + + p 2 x 2 

- Location of peak more important than peak value, „ 2 

so need not match functional form precisely | 3 

~o 

Q_ 

• With regression analysis of training dataset, |- 
can establish prior distributions of set of 
P values 

• Drawing from these priors, can use Bayes’ < 

Ratio of 1 -sigma Covariance Radius to Miss Distance 

theorem to construct posterior distributions 

-This allows priors to be combined with unfolding 
data from current event 

-Can then estimate log(P c ) from mean values from parameter posteriors 



• ft a.i. solutions 
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Using Frequentist Methods 


• If we refit the line each time we receive a new OCM using frequentist 
methods (/.e. least squares), we would see something like this 
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a.i. solutions 


Estimated Trend After 3 OCMs 
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Using Bayesian Methods 


• If we use Bayesian methods, it is possible to incorporate prior 
information into the estimates 


Estimated Trend Before Data 



Estimated Trend After 2 OCMs 



Estimated Trend After 3 OCMs 


Estimated Trend After 4 OCMs 
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Comparisons Between the Bayesian and 

Frequentist Models 



• Using the Bayesian methods, we can make predictions using only 
two OCMs (though generally these are not particularly informative) 

-This is not possible with the frequentist model 

• The frequentist model fits the points as closely as possible, 
whereas the Bayesian model incorporates prior information, 
compromising between the current and previous data 

• The fits are generally similar, but the Bayesian fit is generally more 
conservative 

-The Bayesian model takes into account the uncertainty of the estimates, thus it 
is less likely to fit the data “too well” 

-As a result, the Bayesian model generally has wider error bounds, which are 
usually more realistic 

-The frequentist approach tends to chase the action, whereas the Bayesian 
approach is more realistically predictive (because it considers prior 
information) 


• I a.i. solutions 
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Methodology Details (1 of 3) 


• We can calculate what is known as the posterior density of the 
parameters given the data 

-p(Ply)«p(ylP)*p(P) 

-Thus, we specify a prior distribution for the beta parameters p(P), update it with 
the data that we have seen p(y|P), and get an updated probability distribution 
of the beta parameters given the data p(P|y) 

• Now, we can force the parabola to open downwards by choosing 
priors the allow only this shape 

- Consider the model Y = p 0 + p^ + p 2 x 2 + £, where £ is the noise in the 
measurement 

- If we force p 0 and p 2 to be negative, this will ensure a downward opening 
parabola will be fit each time and ensure that the vertex be realizable (e.g., not 
have a y-value greater than 1 , which is not possible for a Pc value) 

-This presents one potential hazard with the model: what if the observed data 
actually had the shape of an upward opening parabola? It would be fit with a 
horizontal line, which is not the correct shape 


a.i. 


solutions 
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Methodology Details (2 of 3) 


• The resulting constraints are 

Po<0 

(P,) 2 < 4 p 0 P 2 
P 2 < 0 

• In order to specify these priors, we use truncated Normal 
distributions, so that 

Po ~ Normal(|j 0 ,a 0 2 )l(-°o,0) 

Pi ~ Normal(M 0 ,a 0 2 )l(-2V(p 0 p 2 ), 2V(p 0 p 2 )) 

P 2 - Normal(|j 2 ,a 2 2 )l(0, 00 ) 

• While other prior distributions are possible, we find that the 
truncated normal have the best convergence properties 

-Gamma distributions were also attempted but exhibited high levels of 
autocorrelation and overall slow convergence 


• I a.i. solutions 
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Methodology Details (3 of 3) 



• Assume that e ~ Normal(0, a 2 ) 

• Assume a Gamma prior on the inverse of the variance l/o 2 

-Common practice. 

• Choosing the parameters of these prior distributions 

- Use restricted maximum likelihood to estimate downward opening parabolas 
on a set of test data (we examined over 1000 events) 

- Collect all of the betas from the fits 

- Find parameters of a truncated normal distribution that is close to the observed 
distribution of each parameter by matching quantiles 


• ft a.i. solutions 
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Bayesian Vertex Model: 
Model Performance Investigation 



• Conjunction data archive assembled for 2013-14 for well-populated 
orbit regime 

- Perigee height between 500 and 750 km and eccentricity < 0.25 
-Thousands of events per year 

• Use part of 2013 data to “train” model — set prior distribution 
coefficients 

• Use 2014 data as validation dataset 

• Segregate performance results 

- First, by total number of data points (CDMs) in the event 

• Data-poor events may perform worse than data-rich ones 
-Second, by data point number 

• How does model perform after point 3 versus point 6 or 10? 


• ft a.i. solutions 
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• Probably want at least 50 events 
surveyed to feel confident about 
model performance conclusions 

• This achieved only for event 
sizes smaller than 14 data points 

• Should focus on performance 
results for these shorter events— 
sampling more plentiful 


• ft a.i. solutions 


# of Events 


Bayesian Vertex Model: 
LE02: Data Density 
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Bayesian Vertex Model: 
Mean Peak Estimation Error 



mean(Y - Yhat) for all the events of each 
size 

Value becomes unstable beginning at 
event sizes of about 13 observations 

Stable region shows mean values 
ranging from around 0 +- half an order of 
magnitude 

Model is biased but biased in a favorable 
direction 

- Overpredicting leads to conservative safety- 
of-flight decisions — better than the reverse 




20 


16 


MaxPc Prediction Bias (Log10[Observed/Expected]) 


4 6 8 10 12 14 16 18 

Event Size (# of Data Points) 

MaxPc Prediction Bias (Log10[Observed/Expected]) 


3 4 5 6 7 8 9 10 11 12 

Event Size (# of Data Points) 


12 


10 


02 


0.1 


0 


a.i. solutions 
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Bayesian Vertex Model: 
50 th and 95 th percentile Peak 
Absolute Residual Errors 


Focus on more stable region (event 
sizes of 13 or fewer points) 

At the 50 th percentile all of that area is 
less than 0.5 of an order of magnitude 

-An acceptable result 

At the 95 th percentile, that area varies 
between 0.5 and 3 orders of magnitude 

- Probably not an acceptable result 

Model probably not useful for peak 
prediction 

However, could still be useful for 
predicting whether peak has occurred 


SOile Pc Prediction Log10(Residual Absolute Value) 
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a.i. solutions 
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Bayesian Vertex Model: 
Peak Prediction Performance 


• Operational question: has the event 
reached its peak Pc value? 


• Plot at right shows, for all events of a 
certain size after a certain data point, 
the percent correct peak predictions 

- % of the time the model indicates the peak 
has already passed, and in fact it has 

• In region of interest (< 14 data points), 
performance always better than 50% 
once half the event points received 

- Performance moves to 80-100% as number 
of points reaches total event size 

• However, difficult to use result, since # 
of points not known in advance 
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- Examine predictive force at “times to TCA” of 
operational interest 


a.i. 


solutions 
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Bayesian Vertex Model: 
Peak Prediction Performance (cont’d) 


• Examine situation at typical maneuver 
planning and commit times 


-4, 3, 2, and 1 days before TCA 

• Blue bars show percentage of correct 
before/after peak predictions at these 
time points 

• Yellow bars show number of events 
for which prediction was possible 

-At least two points needed 
- MCMC fails to converge occasionally 

• Not stunning performance, but could 
be an operational tool of some utility 
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• ft a.i. solutions 
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Conclusion/Future Work 


• A simple statistical model shows operational promise in 
determining whether the peak Pc value has occurred 

• Additional areas requiring exploration 

- Event Pc histories need categorization 

• May be that algorithm performs well only in “obvious” cases; may not be helpful 
more ambiguous situations where greater operational need 

- Different overall functional forms may yield better results 

• For instance, the log probabilities of collision are effectively bounded between -10 
and 0, suggesting a different distribution (Beta) may be more appropriate 

- Other modeling paradigms 

• Other ways of borrowing information, e.g. mixed models 

- Longitudinal data analysis, because the observations are repeated 
measurements on different events 


• ft a.i. solutions 
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